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(54) Apparatus and method for modifying a compressed video signal 



(57) A compressed video stream (10) that exhibits 
temporal redundancy (e.g.. an MPEG-compliant video 
stream) is decompressed into an uncompressed do- 
main representation (14). and the uncompressed do- 
main representation is modified without substantially 
changing spatial relationships in the representation 
(18). Such modifications might include adding water- 
marks to the uncompressed domain representation, re- 
moving watermarks from the uncompressed domain 



representation, altering watermarks in the uncom- 
pressed domain representation, and adjusting hue and 
intensity in the uncompressed domain representation. 
The modified representation is then recompressed us- 
ing the same motion vectors that were used to produce 
the original video stream (20). Thus, the uncompressed 
domain representation is recompressed without recom- 
puting new motion vectors! thereby avoiding computa- 
tionally intensive processing. 
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Description 

(0001] The invention relates to data compression ac- 
cording to a motion-compensated prediction scheme 
that exploits temporal redundancy. The invention also 5 
relates to copy protection of a compressed data stream. 
[0002] High capacity disc players such as DVD play- 
ers can provide access to wide screen movies having, 
high quality video images and high fidelity sound. The 
high quality format of the DVD player brings theatre-like io 
entertainment to the home. 

[0003] A full-length movie may be stored on a single 
DVD. This is made possible because the DVD has a 
high storage capacity. A single DVD can presently store 
several gigabytes of data. This is also made possible '5 
because video sequences are compressed according 
to an MPEG-based compression scheme. Modern com- 
pression schemes such as MPEG can reduce data stor- 
age requirements by factors of 15 to 80 without a con- 
siderable loss in the quality of the reconstructed video 20 
sequences. 

[0004] However, the advent of read/write DVD drives 
for computers has made it possible to copy the DVD re- 
leases onto blank DVDs, thereby providing the oppor- 
tunity to make unauthorised copies of the DVD releases. 
In the absence of a copy protection scheme, quality of 
these unauthorised copies is comparable to the quality 
of the DVD releases sold in stores. Therefore, unauthor- 
ised copying and distribution of DVD releases pose a 
challenge to artists, producers and copyrights owners 30 
who want to protect their capital investments and intel- 
lectual property rights. 

[0005] Different copy protection schemes are availa- 
ble to thwart unauthorised copying and distribution of 
the DVD releases. For example, resilient watermarks 35 
can be "imprinted" in a compressed video sequence. A 
DVD player can then use the watermark to determine 
whether a disc containing an unauthorised copy is being 
played. The watermark might indicate whether the disc 
is compatible with the video sequence stored thereon. *o 
if the watermark indicates that the video sequence 
should only exist on stamped disks, the DVD player de- 
tects an incompatible "written" disc and shuts down. 
[0006] However, modifying the video sequence by 
adding the watermark can be problematic, especially if **5 
the video sequence has already been compressed into 
an MPEG-compliant video stream. Decompressing the 
MPEG-compliant video stream into spatial-pixel domain 
representations, modifying the pixel domain represen- 
tations, and then re-compressing the modified pixel do- 50 
main representations back into an MPEG-compliant vid- 
eo stream is computationally intensive. 
[0007] There might also be a need to alter an existing 
watermark after the watermark has been added to the 
video sequence. There might even be a need to remove 55 
the existing watermark from the video sequence. Here 
too. modifying the video sequence is computationally in- 
tensive, especially if the video sequence has already 



been compressed into an MPEG-compliant video 
stream. 

[0008] The present invention seeks to provide a way 
of modifying an MPEG-compliant video stream. 
[0009] According to an aspect of the present invention 
there is provided apparatus as specified in claim 1. 
[0010] In the preferred embodiment, a first data 
stream that was compressed according to a motion- 
compensated prediction scheme such as MPEG is de- 
compressed into an uncompressed domain representa- 
tion. The uncompressed domain representation is then 
modified without substantially changing spatial relation- 
ships in the representation. The modified representation 
is then recompressed into a second data stream. The 
motion vectors in the first data stream are used in the 
second data, stream. 

[0011] An embodiment of the present invention is de- 
scribed, by way of example only, with reference to the 
accompanying drawings, in which: 

Figure 1a is an illustration of exemplary frames of 
an MPEG compliant video stream, the frames being 
shown in a display order; 

Figure 1b is an illustration of exemplary frames of 
an MPEG-compliant video stream, the frames being 
shown in a coding order: 

Figure 2 is a block diagram of a preferred embodi- 
ment of system: 

Figure 3 is a flowchart of a method of recompress- 
ing a modified spatial pixel domain representation: 
Figure 4 is a block diagram of an embodiment of 
computer for modifying an MPEG compliant video 
stream: and 

Figure 5 is a flowchart of an alternative method of 
modifying a video stream. 

[0012] As shown in the drawings for purposes of illus- 
tration, the preferred embodiment is described in con- 
nection with a system for modifying a video stream such 
as an MPEG-compliant video stream. The system al- 
lows the video stream to be modified in a manner that 
is less computationally intensive than simply decom- 
pressing the MPEG video stream into an uncompressed 
representation, modifying the video in the uncom- 
pressed representation, and then generating a com- 
pletely new MPEG-compliant video stream including 
new motion vectors. Consequently, the system allows 
watermarks to be added to. or removed from, a video 
stream in a less computationally intensive manner. 
[0013] In the paragraphs that follow, a description of 
the preferred embodiment of system will be provided. 
The system will be described mainly in connection with 
a spatial pixel domain. First, however, descriptions of a 
video sequence in the spatial domain and an MPEG- 
compliant video stream will be provided. 
[0014] In the spatial-pixel domain, a video sequence 
may be described as a hierarchy of units. A video se- 
quence is made up of a group of pictures ("GOPs"}. 
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Each GOP is made up of a number of contiguous pic- 
tures, and each picture is made up of a number of slices. 
Each slice includes a number of macroblocks. and each 
macroblock includes a block of pixels. Each MPEG mac- 
roblock typically contains one or more 8x8 blocks of pix- 
els. 

[0015] MPEG is a block motion-compensated predic- 
tion scheme that reduces both temporal and spatial re- 
dundancies inherent in the video sequence. MPEG cod- 
ing is generally performed as follows. The pictures of a 
video sequence are coded as intraframes ("I frames"), 
forward prediction frames ("P frames") and bi-direction- 
al prediction frames ("B frames"). Each GOP typically 
refreshes the temporal prediction by coding the first pic- 
ture in the GOP as an i frame. The first picture in the 
GOP is scanned in a left-to-right, top-to-bottom fashion. 
Every block (i.e.. 8x8 block) is coded. No prediction is 
used in the I frame. 

[0016] The remaining pictures in the GOP are typical- 
ly coded with interframe (predictive) coding techniques. 
A simplified description is as follows. The blocks in each 
of the remaining pictures of the GOP are scanned in a 
left-to-right, top-to-bottom fashion. A block is compared 
to similar sized sections of adjacent pictures. If a good 
match for the block is found, only the difference between 
the block and the matching section and the vector point- 
ing to the matching section (i.e.. the motion vector) are 
processed. The difference between the block and the 
matching section is commonly referred to as a "differ- 
ence block." Performing this motion analysis is compu- 
tationally intensive. 

[0017] Typically, one motion vector is specified for 
each forward-predicted or backward-predicted block, 
and two motion vectors are specified for each bidirec- 
tionally predicted block. Thus, each P frame may include 
motion vectors that are related to a previous frame, 
while each B frame may include forward and backward 
motion vectors that are related to adjacent frames 
[0018] In P frames, each block may be coded using 
forward predictive or intraframe coding techniques. In B 
frames, each block may be coded using forward predic- 
tive coding techniques, backward predictive coding 
techniques, bi-directional predictive coding techniques, 
or intraframe coding techniques. 

[0019] The MPEG syntax allows a GOP to contain a 
flexible number of pictures, but typical GOP sizes range 
between fifteen and thirty five frames. The I. P .*nd E 
frames for each GOP are arranged in flexible A 
typical IPB pattern in a display order is show?' •* - 
1a. and a typical IPB pattern in a coding ord~- • • * .vn 
in Figure 1 b. Data is placed in a video stream : : ^ ) 
to the coding order. 

[0020] The original intraframe blocks and j --- -- - 

(i.e.. residual interframe) blocks are then Iran-' 
from the spatial domain to the frequency domain : • 
uses Discrete Cosine Transform ("DCT") cod"* : 
transform the intraframe and residual interframe be 
into 8x8 blocks of DCT coefficients. 



[0021] The resulting DCT coefficients are quantized. 
Quantization usually results in a sparse representation 
of the data, i.e., usually most of the amplitudes of the 
quantized DCT coefficients are equal to zero. The quan- 
5 tized DCT coefficients are run-length encoded and then 
variable-length encoded using Huffmann coding. The 
motion vectors are also compressed using variable 
length coding techniques. 

[0022] An MPEG-compliant video stream includes 
10 encoded DCT coefficients and encoded motion vectors. 
The DCT coefficients and motion vectors are grouped 
in GOPs. The video stream also includes header infor- 
mation for each of the GOPs. 

[0023] Figure 2 shows a system 12 for modifying the 
is MPEG-compliant video stream 10. The MPEG-compli- 
ant video stream 10 is supplied to an MPEG decoder 
14. which completely decompresses the video stream 
10 into a pixel domain representation of the MPEG-en- 
coded video sequence. The decoder 14 decompresses 
20 the video stream 10 in a conventional manner. The video 
stream 10 is decoded into quantized coefficients and 
motion vectors, the quantized coefficients are recon- 
structed, and the motion vectors and reconstructed DCT 
coefficients are used to reconstruct the pixel domain 
25 representation of the video sequence. In addition to per- 
forming the conventional decoding of the video stream 
10. the decoder 14 stores the motion vectors in a buffer 
1 6 for later use. 

[0024] A conventional video editor 18 can then modify 

30 the pixel domain representation of the video sequence 
However, the modification should be made without sub- 
stantially changing spatial relationships in the pixel do- 
main representation. Such modifications may include. 
* without limitation, the following: adjusting the intensity 

35 and hue in the pixel domain representation: adding an 
artifact to the pixel domain representation, removing an 
existing artifact from the pixel domain representation, 
reducing noise, and altering an existing artifact in the 
pixel domain representation. 

•to [0025] For example, an existing watermark could be 
removed from a video sequence. Or. a watermark could 
be added to discourage unauthorised copying of the vid- 
eo sequence. A watermark could be added to the pixel 
domain representation in a conventional manner. The 

*5 watermark could be distributed in the representation as 
noise that is invisible to the viewer when the video se- 
quence is being played back. However, the watermark 
can be detected by a copy protection-enabled system 
The watermark contains information indicating whether 

5'j a disc is compatible with the video sequences stored 
thereon. If a disc containing an unauthorised copy of the 
video sequence is inserted into the copy protection en- 
abled player, and the watermark indicates that the video 
sequence should only exist on a stamped disc, the play- 
er will not play the disc. Instead, it might shut down 
[0026] A re-encoder 20 recompresses the modified 
pixel domain representation back into an MPEG-com- 
pliant video stream. The re-encoder 20 does not perform 
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motion analysis on the modified pixel domain represen- 
tation. Instead, the re-encoder 20 re-uses the motion 
vectors stored in the buffer 16. Using the buffered mo- 
tion vectors, the re-encoder 20 generates an MPEG- 
compliant video stream 22 for the modified pixel repre- 
sentation (the "modified" video stream 22). Thus, rec- 
ompression of the modified pixel domain representation 
is less computationally intensive then the compression 
that was used to produce the original video stream 10. 
Additionally, the recompression is about as computa- 
tionally intensive as the decompression performed by 
the decoder 14. The modified video stream 22 may then 
be used to create a master DVD. or the modified video 
stream 22 may be stored on a disc. 
[0027] Reference is now made to Figure 3. which 
shows how a P or B frame is coded by the re-encoder 
20. The re-encoder 20 scans blocks in the picture (block 
100) and accesses the corresponding motion vectors 
stored in the buffer 16 (block 102). The re-encoder uses 
the motion vectors to find the matching sections in ad- 
jacent frames and to recompute the difference blocks 
(block 104). In this manner, the motion vectors are re- 
used. Thus, the difference blocks are recomputed with- 
out having to recompute the motion vectors. 
[0028] New DCT coefficients are recomputed for the 
macroblocks (block 106). The new DCT coefficients are 
quantized (block 108). and the quantized coefficients 
are run length encoded and variable length encoded 
(block 110). The buffered motion vectors are variable 
length encoded (block 112). Header information is then 
added (block 114) to produce the modified video stream 
22. 

[0029] If the same sequence of frames is kept, the 
headers and certain I. P and B frames may also be buff- 
ered in the buffer 16 and re-used during re-compres- 
sion. Since the re-encoder 20 does not re-compute the 
motion vectors, it produces an MPEG-compliant stream 
in a less computationally intensive manner. 
[0030] The system 12 may be realised by a combina- 
tion of hardware and software components, such as a 
software-based video editor and a hardware-based en- 
coder and re-encoder. In the alternative, the system may 
be implemented in software. 

[0031] A software implementation is shown in Figure 
4. A computer 200 includes a processor 202 and mem- 
ory 204 that is encoded with a plurality of executable 
instructions. When executed, the instructions instruct 
the processor 202 to access the video stream iv -e g . 
from a peripheral device, a hard drive, the lnKr-~ • oind 
then perform the decompression, video edmr. '; : n j re- 
compression described above. The memory J - m.*v 
also provide the buffering. The modified vidtr ■•- 
22 may then be sent to a data storage device > ■' - : 
a hard drive or DVD player) or an output card 2\ - - : 
a modem or network card) for transmission to .*ip. 
computer. 

[0032] Thus disclosed is a system that decompresses 
a video stream into a spatial domain representation, al- 



lows the spatial domain representation to be modified, 
and then re-uses motion vectors to recompress the 
modified spatial domain representation. Recompres- 
sion of the modified pixel domain representation is less 
6 computationally intensive then the compression that 
produces the original video stream, and it is about as 
computationally intensive as the decompression that 
produces the pixel domain representation. Thus, the 
system allows a video stream to be modified in a manner 
io that is more computationally efficient than simply de- 
compressing a video stream, making modifications in 
the spatial pixel domain, and then generating a com- 
pletely new video stream including new motion vectors. 
Consequently, watermarks can be added to an MPEG- 
'S compliant video stream, removed from the video 
stream, or altered in a less computationally intensive 
manner. Similarly, the hue and intensity of the video 
stream can be adjusted in a less computationally inten- 
sive manner. 

20 [0033] When making a modification such as adjusting 
hue or introducing a small amount of spatial change, the 
buffered motion vectors might no longer be ideal. Con- 
sequently, the difference blocks might not code quite as 
well. However, the processing efficiency might justify the 

2$ slight loss in quality. 

[0034] The system could be applied to various MPEG 
coding standards such as MPEG-1 and MPEG-2. as 
well as other coding standards that rely on motion-com- 
pensation (e.g.. motion JPEG. H.261 . H.263). However. 

30 the method is not limited to such coding standards. The 
example of the MPEG-compliant video stream 10 was 
provided merely to facilitate an understanding of the de- 
scribed embodiment. 

[0035] The method could be used in conjunction with 
35 any compression scheme that uses motion vectors or 
equivalents thereof. 

[0036] The method is not limited to DVD players. The 
example of the DVD player was also provided to facili- 
tate an understanding of the preferred embodiment. 

JO [0037] Moreover, the invention is not limited to the 
specific embodiments described and illustrated above. 
For example, the re-encoder could generate the motion 
vectors independently of the decoder. Such a re-encod- 
er would parse the video stream to regenerate the mo- 

-5 tion vectors. This would eliminate the buffer 16 and allow 
an off-the-shelf decoder to be used. 
[0038] The method is not limited to a spatial pixel do- 
main representation. For instance, the video sequence 
may be modified in a frequency domain. Referring to 
Figure 5. the video stream may be decompressed into 
a frequency domain representation by reversing the var- 
iable length encoding, the run length encoding, etc 
(block 300). The frequency domain representation may 
then be modified by directly modifying the reconstructed 
DCT coefficients (block 302). The modified frequency 
representation may then be recompressed into a mod- 
ified stream by quantizing the DCT coefficients {block 
304). and run length encoding and variable length en- 
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coding the quantized coefficients (block 306). The mo- 
tion vectors are not modified: they are simply variable 
length coded (block 308). Thus, the motion vectors in 
the original video stream are used in the modified 
stream. 

[0039] Thus, the steps involved in performing the de- 
compression and recompression may or may not in- 
clude taking DCT and inverse DCI transforms. Instead, 
the decompression and compression steps will depend 
upon the type of domain that is edited. 
J0040] The disclosures in United States patent appli- 
cation no. 09/198.052. from which this application 
claims priority, and in the abstract accompanying this 
application are incorporated herein by reference. 

Claims 



domain representation and wherein the re-encoder 
is operable to compress a modified frequency do- 
main representation. 

5 6. The apparatus of claim 1. comprising a buffer (16) 
for buffering the motion vectors generated during 
decompression of the first data stream, the re-en- 
coder being operable to use the buffered motion 
vectors for the generation of the second stream. 

70 

7. The apparatus of claim 1. wherein the decoder is 
operable to perform decoding according to an 
MPEG standard, wherein the re-encoder is opera- 
ble to perform encoding according to an MPEG 
»5 standard, and wherein the first and second data 

streams are MPEG-compliant video streams. 



1. Apparatus (12) for allowing a first motion vector- 
compliant data stream (10) to be modified, the ap- 20 
paratus comprising: 

a decoder (14) responsive to the first stream, 
for providing an uncompressed domain repre- 
sentation, and a plurality of motion vectors dur- 2b 
ing decompression of the first data stream; and 
a re-encoder (20) responsive to the motion vec- 
• tors and a modified uncompressed domain rep- 
resentation, for providing a second motion vec- 
tor-compliant data stream (22). the re-encoder 30 
using the plurality of motion vectors in the sec- 
ond stream. 



2. The apparatus of claim 1 . comprising a video editor 
(20) for modifying the uncompressed representa- 36 
tion. the video editor being operable to provide the 
modified representation to the re-encoder. 

3. The apparatus of claim 2. wherein the decoder, vid- 
eo editor and re-encoder are realised by a computer *o 
(200) including a processor (202) and memory 
(204). the memory being encoded with a plurality of 
instructions that, when executed, instruct the proc- 
essor to decompress the stream into the uncom- 
pressed domain representation, allow the uncom- -'5 
pressed domain representation to be modified, and 
re-compress the modified representation into the 
modified data stream. 



4. The apparatus of claim 1. wherein the decoder is 50 
operable to use the plurality of motion vectors to 
provide an uncompressed spatial domain represen- 
tation, and wherein the re-encoder is operable to 
use the plurality of motion vectors to compress a 
modified spatial domain representation. 

5. The apparatus of claim 1. wherein the decoder is 
operable to provide an uncompressed frequency 
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FIGURE 5 
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